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SUMMARY 


This paper describes the development of a life-cycle cost (LCC) estimating methodology for air 
traffic control Decision Support Tools (DSTs) under development by the National Aeronautics and 
Space Administration (NASA), using a combination of parametric, analogy, and expert opinion 
methods. There is no one standard methodology and technique that is used by NASA or by the 
Federal Aviation Administration (FAA) for LCC estimation of prospective Decision Support Tools. 
Some of the frequently used methodologies include bottom-up, analogy, top-down, parametric, 
expert judgement, and Parkinson’s Law. The developed LCC estimating methodology can be 
visualized as a three-dimensional matrix where the three axes represent coverage, estimation, and 
timing. This paper focuses on the three characteristics of this methodology that correspond to the 
three axes. 


INTRODUCTION 


Insufficient capacity, limited access, and excessive restrictions have escalated operation costs 
and delay for all users of the National Airspace System (NAS). The National Aeronautics and Space 
Administration (NASA) Advanced Air Transportation Technologies (AATT) project is developing 
Decision Support Tools (DSTs) that are computer-based analysis, prediction, and display aids for air 
traffic controllers. These tools will facilitate substantial increases in the effectiveness of national and 
global air transportation systems. The AATT project is responsible for defining, exploring, and 
developing advanced air traffic management system concepts through preproduction maturity. From 
there the technology is transferred to the Federal Aviation Administration (FAA), which, if it 
decides to deploy the DST, carries out full-scale development and deployment. During the course of 
the NASA research and development (R&D) effort, NASA conducts life-cycle cost-benefit studies 
at several stages of maturity, to indicate whether the DST will have a positive return on investment if 
deployed by the FAA. These studies require a fairly accurate assessment of the life-cycle cost (LCC) 
of a DST. 

LCCs are the sums of every cost incurred for a particular system over its lifetime, except for 
sunk costs (ref. 1). LCCs usually include R&D, fabrication and testing, operation, maintenance, and 
disposal costs. To date, there is no standard LCC estimating methodology and technique that is used 
by NASA or the FAA for air traffic management systems. Some of the more frequently employed 
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methodologies include bottom-up, analogous, expert judgement, and parametric estimating. 
Recently, parametric techniques have gained popularity because they can provide reliable estimates 
that are generated at a lower cost and shorter cycle time than other traditional techniques (ref. 2). 

The AATT DSTs are software tools on Commercial Off-the-Shelf (COTS) hardware equipment. 
The LCC of the DSTs requires assessing both software and nonsoftware costs. Existing software 
cost estimating models, such as COCOMO (Constructive COst MOdel), fit only a portion of the 
LCC estimation needs. Because of the differences in software cost estimating models and the large 
uncertainty associated with software cost estimation, there is also a need to use at least two software 
cost estimating methods to verify the software cost quantification. The methodology presented in 
this paper was developed to satisfy these needs in estimating the LCC of NASA-developed DSTs. It 
is important to note that developing the LCC estimation methodology is only one of many steps in a 
LCC analysis (ref. 3). 


METHODOLOGY 


When assessing the LCC for a system, three cost characteristics need to be addressed — 
consideration of all cost types (coverage), quantification of these costs (estimation), and establishing 
timing of these costs (LCC phase). The LCC methodology can be visualized in figure 1, which 
illustrates it as a three-dimensional matrix where the three axes represent coverage, estimation, and 
LCC phase. The outline of the methodology is provided in the following discussion of the three 
axes. 


Engineering 
Adaptation Data 
Production 
Reviews and Audits 
NAS Integration 
NAS Information Security 
Staffing 


COST 

ESTIMATION 



NASA Development Costs 

Annual Program Costs 
Initial Costs at NASA Demo Site ' 

Initial Costs at all other FAA Sites 

Annual Costs at all Sites " 

Intermittent Costs at all Sites 

Termination Costs at all Sites ' 


Figure 1. Cost characteristics for LCC assessments. 
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Coverage Axis 


The elements on the coverage axis ensure that all types of costs associated with the life cycle of 
the system are included. Various types of coverage structures have been used in LCC analyses 
work breakdown structure, cost element structure, and subdivisions of work structure. In application 
of the methodology developed in this paper, the cost element structure is used for the coverage axis. 
The cost elements and cost factors reflect the accounting subdivisions of program costs, and include 
research, development, transfer, operations, and maintenance. Some of these cost factors are shown 
in figure 1. Given the knowledge base at the time of the assessment, every effort should be made to 
make this a comprehensive coverage of all cost factors. It is best to have a large list of cost factors so 
that they can be used as a checklist to help analysts achieve a comprehensive assessment. 

Estimation Axis 


The elements on this axis represent the assessment methodologies for the various cost elements 
on the coverage axis. Each element on the coverage axis must have at least one corresponding 
element on the estimation axis. In this paper, the cost factors are either not applicable (hence, not 
assessed) or they are assessed in one of three methodologies — software, hardware, or other (ad hoc). 
Many software and hardware cost estimating techniques and models are available. In this analysis, 
the NASA DST software costs were estimated using an internally developed Activity-Based Cost 
(ABC) model and the COCOMO II model. The cost of the COTS hardware was assessed based on 
manufacturer quotes. The rest of the cost factors were assessed using a combination of analogy, 
parametric, and expert opinion methodologies based on available knowledge. 

LCC Phase Axis 

Every quantified cost on the coverage axis occurs at some point in the life cycle. The timing of 
these costs is indicated on the LCC phase axis. During conceptual designs, the timing may be known 
only by phase (concept, demonstration and validation, technology transfer, production, deployment, 
operations and support, retirement, etc.). If the cost estimating process includes a work breakdown 
structure, the timing may be known more precisely (by month). Evaluation of the timing of the costs 
is important for Net Present Value (NPV) calculation. Figure 1 shows the timing of costs broken 
down into categories that were useful for the LCC assessment of NASA DSTs. 

As part of the LCC methodology, it is also important to determine the base year of analysis, the 
economic service life of the system, and the discount rate for NPV calculations. The base year is 
usually the current year or the year in which the first cost associated with an alternative is incurred. 
In an economic analysis, all costs are discounted to the base year. If the estimated costs are not 
assessed at the base-year dollar values, then conversion to base-year costs requires knowledge of the 
deflation/inflation rate. 
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LCC PHASE 


To estimate the DST life-cycle costs, we must first understand the sequence of events in a DST 
program’s life cycle. Figure 2 schematically describes a road map of a DST’s life cycle from NASA 
R&D to the end of the DST program. 
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Figure 2. Example of a DST life cycle. 
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NASA is developing the DSTs from Technical Readiness Level (TRL) 1 (Basic Technology 
Research) to TRL 6 (Prototype System Demonstrated in Relevant Environment). Each of these 
DSTs is being developed at a demonstration site in the United States. At completion of TRL 6, the 
FAA has the option to pick up the NASA-developed technology. If the FAA chooses to do so, then a 
technology transfer from NASA to FAA occurs. The FAA then develops the DST to a state of 
operational readiness at the demonstration site, with an initial daily use (IDU) leading to a planned 
capability available (PCA). The FAA may choose to deploy the DST at other sites; this is shown in 
figure 2 as the second-site PCA through the last-site PCA. The economic service life of the DST was 
determined from reference 4 — it is the period of time during which the DST is expected to provide a 
positive benefit. After the economic service life, one by one the DST is taken out of service from 
each site, and the FAA's program ends soon after the removal of the DST from the last site. 


Based on the timeline of events in figure 2, DST costs were categorized into one-time-only 
program costs, recurring annual costs, recurring intermittent costs, initial costs specific to certain 
sites, and termination costs for each site. For clarification, the following symbols are assigned to 
each category. 


• One-time-only costs: OC 

• Annual program costs: AP 

• Initial costs at the first DST site: II 

• Initial costs at the i * DST site (i > 1): 12 

• Annual costs at all DST sites: AC 

• Intermittent costs at all DST sites: IC 

• Termination costs at all DST sites: TC 
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After all the cost factors were associated with a LCC phase (see table 1), the timing (year of 
occurrence) of the costs related to each cost factor was determined based on the timing of the LCC 
phases. The methodology to establish the timing of the LCC phases was based on information from 
previous implementations of similar DSTs by the FAA. 

COVERAGE OF COSTS 


In this paper, the cost element structure has been used for the coverage axis. The methodology 
requires a two-level hierarchical arrangement of cost factors and cost elements. One or more cost 
factors combine to make a cost element. 

A list of generic NASA and FAA project cost elements and factors over the life cycle of a DST 
was studied. The chosen factors were based partially on information provided in reference 4. They 
were representative of what may be needed by NASA and the FAA to fund and implement a DST 
acquisition program. The list was intended to be comprehensive so that it applied to all the DSTs. 
Consequently, not all cost elements and cost factors considered were applicable to every DST. Only 
those that were quantified for the example DST are listed in table 1. Table 1 also shows the LCC 
phase and the cost estimating models used. The abbreviations for the cost estimating models follow: 

• Software-related cost estimating: S 

• Hardware-related cost estimating: H 

• Other (ad hoc) cost estimating: O 


Table 1 . Cost elements and factors quantified in an example DST life-cycle cost evaluation 


Coverage of costs 

Estimation 

LCC phase 

Cost element 

Cost factors 

NASA's R&D 

R&D 

SorO 

oc 

FAA's program 
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0 
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Program management 

0 
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O 

oc 
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12 
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11 
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TT production 
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11 
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s 

11 

DST software 
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12 
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0 
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s 
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Physical integration 
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H 
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Power system 

0 

AC 


T elecommunications 

0 

AC 
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Table 1 . (Continued) 
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Following is a discussion of these various cost elements: 

• NASA's R&D costs includes all applicable cost starting from TRL 1 to TRL 6 completion. In 
this study, NASA's sunk costs prior to the start of the AATT project were not included in the 
assessment (ref. 1). 

• FAA's program management covers the planning and monitoring of all tasks and resources over 
the entire life of the DST at all sites. 

• Activities external to the program (not shown in table 1 because they are not applicable to the 
example DST) are those that may be needed for rulemaking and interfacing with other 
organizations in fielding the DST. 

• Facilities costs (not shown in table 1 because they are not applicable to the example DST), if 
required, cover the architecture, engineering, and construction of special facilities. 

• The technology transfer cost factor covers NASA and FAA costs related to transfer of the DST 
technology from NASA to the FAA. 

• DST software development covers FAA's development, engineering, and production of the DST 
from TRL 6 completion to PCA. 

• Physical integration concerns the integration of the DST into the physical operational 
environment by the FAA. Cost factors include acquisition of real estate or space, engineering for 
environmental compliance, energy conservation, and noise abatement. 

• Functional integration is related to the interface requirements associated with integrating the 
DST into the operating NAS air traffic control and air navigation systems. 

• Human integration costs are due to requirements and standards that ensure that the DSTs are 
designed for the air traffic controllers that will operate it and the human workforce that will 
maintain it. The cost factors relate to safety, training, staffing levels, and personnel skills. 

• Security ensures that the DST does not compromise NAS information or personnel security. Cost 
factors related to maintaining DST physical security are also part of this cost element. 

• In-service support includes cost factors to define supportability requirements associated with 
maintenance, staffing, supply support, training, etc. 

• Test and evaluation relates to all test and evaluation requirements prior to the operational tests 
for the DST. Cost factors include test plans, procedures, reports, equipment/tools, simulations, 
staff, etc. 

• Initial operational test and evaluation (IOT&E) is similar to test and evaluation, but includes 
only the operational tests. 
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• Configuration management is related to identifying and documenting the functional and physical 
characteristics of the DST, controlling changes to DST characteristics, recording and reporting 
change processing and implementation status, and verifying compliance with requirements. 

• Quality assurance cost factors are those that are applicable because of the definition of quality 
assurance requirements. 

• Implementation costs are those that are caused by the requirements related to transitioning from 
the current capability to the new DST capability so as not to disrupt ongoing NAS operations. 

• In-service management costs are related to monitoring, assessing, and optimizing DST 
performance, and planning for major upgrades. 

• Sustainment engineering is related to maintaining the DST with bug fixes, software 
enhancements, operating system upgrades, or replacing obsolete or failed hardware components. 

• Program support services are related to the activities of the FAA’s contractor program 
management and technical support. 

• Operations and maintenance (O&M) resources required by the FAA for a fielded DST are the 
relevant cost factors for this cost element. 

• Terminations cost factors relate to the disposal of hardware and software along with 
reintegration of the affected systems. 

The cost factors that were classified as "not applicable" on the cost estimation axis belonged to 

one of the following four categories: 

• Not required (e.g., DSTs did not require physical integration with roads or sewage). 

• Not quantified because the baseline resources were sufficient, so that additional resources were 
not required (e.g., additional real estate). 

• Not quantified based on study assumptions (e.g., the FAA’s contract reprocurement was 
assumed to be part of the FAA program management costs). 

• Not quantified because they were assumed to be small cost contributors (e.g., the cost of heating, 
cooling, or air conditioning). 
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COST ESTIMATION 


The costs of the remaining cost factors (see table 1) must be estimated. Every remaining cost 
factor was estimated using some cost estimating models — some were part of another cost factor 
(subset), others were estimated along with other cost factors (grouped), while others were estimated 
individually (singleton). The cost estimating models were either software related (software 
development, adaptation, maintenance, and enhancement), hardware related, or ad hoc models. The 
cost estimating model used for each applicable cost factor is also shown in table 1. 

Software-Related Cost Estimation 

As seen from table 1, most of the cost factors were estimated using the software cost estimating 
models. This is expected because the DSTs are software tools that use COTS hardware equipment. 

Activity-Based Cost Model 

The activity-based cost model is described briefly here. Through many years of research, Jones 
(ref. 5) identified 11 activities that comprise a minimum set for activity-based software cost 
estimating. They are chosen based on their high frequency of occurrence during software projects 
and are assumed to be the base list for the ABC model. They are: requirements, prototyping, design 
and specifications, design inspections, coding, code inspections, change management and 
configuration control, testing, user documentation and project documentation, project management, 
and maintenance and enhancement. An activity is defined as the sum of the effort needed to 
complete a key milestone or a key deliverable item. The equation for effort required to complete 
each activity takes one of two forms (refs. 5 and 6): 

Effort^ = size/Prate (1) 

Effort noo)iMl = (Size Power ) x (Size / A Scope) 

The subscript “nominal” is used here because the effort is subjected to the effect of reuse and 
learning. “Size” is the measure of the software project in Lines Of Code (LOC) or Function Points 
(FP). “Power” is a positive real number determined empirically through historical data. “A Scope” 
(Assignment scope) is the amount of work for which one person will be responsible on a software 
project. “P rate” (Production rate) is the amount of work that one person can perform in a standard 
time period, such as a work hour, work week, work month, or work year. Jones recommends a set of 
nominal A Scope, P rate, and Power values by analyzing historical data (ref. 5). 

It has been established that software development costs are influenced by reuse and learning. 
Analysis by Selby (ref. 7) of reuse costs across 3000 reused modules in the NASA Software 
Engineering Laboratory indicates that the reuse cost function is nonlinear (actually, piece-wise 
linear), as seen in figure 3. 
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Ralatlv* Cost vs. Rsus* Factor 



Figure 3. Nonlinear reuse effects. 


Laarning Curva 



Figure 4. Learning curve. 


For the Wright learning curve (a schematic plot is shown in fig. 4), the underlying hypothesis is 
that the direct labor man-hours necessary to complete a unit of production will decrease by a 
constant percentage each time the production quantity is doubled (ref. 8). 

Combining the effects from reuse and learning, we have: 


Effort = Effort . . x F(Reuse, Learning) (3) 

where 


F(Reuse, Learning) = Function of Reuse and Learning as described in figures 3 and 4. 
The total effort in person-months is the sum of effort of all activities. 


COCOMO II 

COCOMO II is a rather complicated and well-documented model. Interested readers are referred 
to Barry Boehm’s book, “Software Engineering Economics” (ref. 9), or COCOMO II handbooks 
(refs. 10 and 1 1) for further information. 

The fundamental equation in COCOMO for the development effort estimate is: 

P^nomlMl = A x (Size) 6 (4) 

where 

= Effort expressed in person-months before adjustment. 

Size = Size of the software product. 

A = A constant (2.94 for the model). 

B = A scale factor that is a function of the project scale drivers (SF,). 
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The scale drivers are chosen because they are a significant source of exponential variation of the 
effort or productivity variation of a project. Meanwhile, cost drivers are used to capture 
characteristics of the software development that affect the effort to complete the project. Cost drivers 
that have a multiplicative effect on predicting effort are called Effort Multipliers (EM,). 

PM Kljuaed = PM nom ,„,x(n i EM,) (5) 

The basic input “Size” in COCOMO is adjusted by a number of factors to account for changes in 
software requirements, reengineering and conversion of code using automated translation, and codes 
from existing software that can be reused. The effort equation does not account for the development 
of software requirements; COCOMO II suggests adding an additional 7 percent to reflect it. 


Calibration of the Models 

A decision, based on an initial study of the software-cost estimating model requirements, was 
made not to use the learning and reuse factors for the ABC model and the three size-adjustment 
factors in the COCOMO D model for this assessment. It is believed that the stringent requirements 
for using these parameters cannot be satisfied. For example, the reuse factor can be used only when 
there is prior similar software. Similarly, use of learning factor assumes that the same personnel 
were working on the same project within a limited period of time. 

Therefore, the two models were then calibrated using information provided by NASA on the 
development of another NASA-developed air traffic control DST. Using the size and cost 
information of the NASA DST, the parameters in the ABC as well as in the COCOMO II models 
were adjusted. The calibrated models then show great agreement over a wide range of software 
sizes, as seen in figure 5 (the error bars in figure 5 represent the range of COCOMO estimates from 
optimistic to pessimistic). 



Figure 5. Comparison of the calibrated ABC and COCOMO II cost estimates. 
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Some of the conceptual DSTs are at a preliminary stage, so even the software requirements are 
not completely specified. So, the software size of the conceptual DSTs was estimated based on 
NASA expert opinion and analogy with previously developed NASA DSTs. The software costs were 
then estimated using these software sizes in the ABC and COCOMO II models. 

Hardware-Related Cost Estimation 

As seen in table 1, only three of the cost factors were estimated using a hardware cost estimating 
model. However, the costs could be substantial and should not be overlooked. In this section, the 
cost model for one of the cost factors, initial hardware acquisition, is described. 

In discussion with NASA, it was determined that the conceptual DSTs would have hardware 
requirements that are very similar to other, previously developed, NASA DSTs (analogy). A detailed 
list of hardware (including backups) needed at each site for a previously developed NASA DST was 
obtained. Three types of hardware were needed for normal operation, namely, network equipment, 
computer processing equipment, and support equipment. 

Using analogy, this list was adjusted to reflect the requirements of the conceptual DST. Because 
all equipment is a COTS-based product, the unit price for all major hardware components was 
obtained from their vendors. The total hardware cost at a conceptual DST site was then calculated. 

C DST = I„1 hard*** (Units x Price) (6) 

where 


C DST = Initial hardware cost for DST at a site. 

The initial hardware acquisition cost is a one-time cost at each site for a conceptual DST. 

Other Cost Estimations 

Most of the cost factors in table 1 needed to be assessed by other, ad hoc methods. For these 
costs (e.g., FAA's program management, technology transfer, telecommunication, power, as well as 
hardware, telecommunication disposition costs, etc.), specialized cost estimating methodologies 
were developed using either parametric, analogy, or expert judgement methods. In this section, the 
Cost Estimating Relationship (CER) for one of the cost factors, FAA's Program Management, is 
described below. 

FAA's personnel costs can be a major component of the ongoing costs of the DST program and 
of complying with government regulations. These are estimated as the product of the quantity of 
labor required and the total compensation paid per unit of labor. Labor requirements are estimated in 
full-time equivalent (FTE) work years, and total compensation as a function of the labor category 
and its corresponding burdened salary rate per FTE per year. 

CaPosT = (52 x 40) x X k Nfte k x Ffed x Salary(Lc k ) (7) 
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where 


Cap DST = Annual program cost for DST in year t. 

Nfte k = Number of full-time equivalents (FTEs) required for each position k. 

Lc k = Labor category for each position k. 

Salary(Lc k ) = Burdened hourly salary rate for Lc k . 

Ffed = Federal employee burdened salary rate fraction, 
t = All years that the DST program is active. 

52 x 40 = 2080 = Number of working hours per year. 

The cost Cap DST applies every year from the start year to the end year of the FAA’s DST 
program. 

CONCLUSIONS 


A structured life-cycle cost estimating methodology was developed for air traffic control DSTs 
under development by NASA. The following three areas were found to be crucial to the 
methodology: coverage of all cost factors, developing/selecting appropriate cost estimating methods 
for different cost factors, and correct timing of all costs. The use of the methodology was illustrated 
with some examples. A key issue is data collection. The parameters in parametric models and cost 
estimating relationships must reflect the attributes of the organizations involved. 
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